A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E).

BACKGROUND: Observational epidemiologic studies provide critical data for the evaluation of the potential effects of environmental, occupational and behavioural exposures on human health. Systematic reviews of these studies play a key role in informing policy and practice. Systematic reviews should incorporate assessments of the risk of bias in results of the included studies. OBJECTIVE: To develop a new tool, Risk Of Bias In Non-randomized Studies - of Exposures (ROBINS-E) to assess risk of bias in estimates from cohort studies of the causal effect of an exposure on an outcome. METHODS AND RESULTS: ROBINS-E was developed by a large group of researchers from diverse research and public health disciplines through a series of working groups, in-person meetings and pilot testing phases. The tool aims to assess the risk of bias in a specific result (exposure effect estimate) from an individual observational study that examines the effect of an exposure on an outcome. A series of preliminary considerations informs the core ROBINS-E assessment, including details of the result being assessed and the causal effect being estimated. The assessment addresses bias within seven domains, through a series of 'signalling questions'. Domain-level judgements about risk of bias are derived from the answers to these questions, then combined to produce an overall risk of bias judgement for the result, together with judgements about the direction of bias. CONCLUSION: ROBINS-E provides a standardized framework for examining potential biases in results from cohort studies. Future work will produce variants of the tool for other epidemiologic study designs (e.g. case-control studies). We believe that ROBINS-E represents an important development in the integration of exposure assessment, evidence synthesis and causal inference.

Expanded Systematic Evidence Map for Hundreds of Per- and Polyfluoroalkyl Substances (PFAS) and Comprehensive PFAS Human Health Dashboard.

BACKGROUND: Per- and polyfluoroalkyl substances (PFAS) encompass a class of chemically and structurally diverse compounds that are extensively used in industry and detected in the environment. The US Environmental Protection Agency (US EPA) 2021 PFAS Strategic Roadmap describes national research plans to address the challenge of PFAS. OBJECTIVES: Systematic Evidence Map (SEM) methods were used to survey and summarize available epidemiological and mammalian bioassay evidence that could inform human health hazard identification for a set of 345 PFAS that were identified by the US EPA's Center for Computational Toxicology and Exposure (CCTE) for in vitro toxicity and toxicokinetic assay testing and through interagency discussions on PFAS of interest. This work builds from the 2022 evidence map that collated evidence on a separate set of ∼150 PFAS. Like our previous work, this SEM does not include PFAS that are the subject of ongoing or completed assessments at the US EPA. METHODS: SEM methods were used to search, screen, and inventory mammalian bioassay and epidemiological literature from peer-reviewed and gray literature sources using manual review and machine-learning software. For each included study, study design details and health end points examined were summarized in interactive web-based literature inventories. Some included studies also underwent study evaluation and detailed extraction of health end point data. All underlying data is publicly available online as interactive visuals with downloadable metadata. RESULTS: More than 13,000 studies were identified from scientific databases. Screening processes identified 121 mammalian bioassay and 111 epidemiological studies that met screening criteria. Epidemiological evidence (available for 12 PFAS) mostly assessed the reproductive, endocrine, developmental, metabolic, cardiovascular, and immune systems. Mammalian bioassay evidence (available for 30 PFAS) commonly assessed effects in the reproductive, whole-body, nervous, and hepatic systems. Overall, 41 PFAS had evidence across mammalian bioassay and epidemiology data streams (roughly 11% of searched chemicals). DISCUSSION: No epidemiological and/or mammalian bioassay evidence were identified for most of the PFAS included in our search. Results from this SEM, our 2022 SEM on ∼150 PFAS, and other PFAS assessment products from the US EPA are compiled into a comprehensive PFAS dashboard that provides researchers and regulators an overview of the current PFAS human health landscape including data gaps and can serve as a scoping tool to facilitate prioritization of PFAS-related research and/or risk assessment activities. https://doi.org/10.1289/EHP13423.

Epidemiology Evidence for Health Effects of 150 per- and Polyfluoroalkyl Substances: A Systematic Evidence Map.

BACKGROUND: Per- and polyfluoroalkyl substances (PFAS) comprise a large class of chemicals with widespread use and persistence in the environment and in humans; however, most of the epidemiology research has focused on a small subset. OBJECTIVES: The aim of this systematic evidence map (SEM) is to summarize the epidemiology evidence on approximately 150 lesser studied PFAS prioritized by the EPA for tiered toxicity testing, facilitating interpretation of those results as well as identification of priorities for risk assessment and data gaps for future research. METHODS: The Populations, Exposure, Comparators, and Outcomes (PECO) criteria were intentionally broad to identify studies of any health effects in humans with information on associations with exposure to the identified PFAS. Systematic review methods were used to search for literature that was screened using machine-learning software and manual review. Studies meeting the PECO criteria underwent quantitative data extraction and evaluation for risk of bias and sensitivity using the Integrated Risk Information System approach. RESULTS: 193 epidemiology studies were identified, which included information on 15 of the PFAS of interest. The most commonly studied health effect categories were metabolic (n=37), endocrine (n=30), cardiovascular (30), female reproductive (n=27), developmental (n=26), immune (n=22), nervous (n=21), male reproductive (n=14), cancer (n=12), and urinary (n=11) effects. In study evaluation, 120 (62%) studies were considered High/Medium confidence for at least one outcome. DISCUSSION: Most of the PFAS in this SEM have little to no epidemiology data available to inform evaluation of potential health effects. Although exposure to the 15 PFAS that had data was fairly low in most studies, these less-studied PFAS may be used as replacements for "legacy" PFAS, leading to potentially greater exposure. It is impractical to generate epidemiology evidence to fill the existing gaps for all potentially relevant PFAS. This SEM highlights some of the important research gaps that currently exist. https://doi.org/10.1289/EHP11185.

Systematic evidence map (SEM) template: Report format and methods used for the US EPA Integrated Risk Information System (IRIS) program, Provisional Peer Reviewed Toxicity Value (PPRTV) program, and other "fit for purpose" literature-based human health analyses.

BACKGROUND: Systematic evidence maps (SEMs) are gaining visibility in environmental health for their utility to serve as problem formulation tools and assist in decision-making, especially for priority setting. SEMs are now routinely prepared as part of the assessment development process for the US Environmental Protection Agency (EPA) Integrated Risk Information System (IRIS) and Provisional Peer Reviewed Toxicity Value (PPRTV) assessments. SEMs can also be prepared to explore the available literature for an individual chemical or groups of chemicals of emerging interest. OBJECTIVES: This document describes the typical methods used to produce SEMs for the IRIS and PPRTV Programs, as well as "fit for purpose" applications using a variety of examples drawn from existing analyses. It is intended to serve as an example base template that can be adapted as needed for the specific SEM. The presented methods include workflows intended to facilitate rapid production. The Populations, Exposures, Comparators and Outcomes (PECO) criteria are typically kept broad to identify mammalian animal bioassay and epidemiological studies that could be informative for human hazard identification. In addition, a variety of supplemental content is tracked, e.g., studies presenting information on in vitro model systems, non-mammalian model systems, exposure-level-only studies in humans, pharmacokinetic models, and absorption, distribution, metabolism, and excretion (ADME). The availability of New Approach Methods (NAMs) evidence is also tracked (e.g., high throughput, transcriptomic, in silico, etc.). Genotoxicity studies may be considered as PECO relevant or supplemental material, depending on the topic and context of the review. Standard systematic review practices (e.g., two independent reviewers per record) and specialized software applications are used to search and screen the literature and may include the use of machine learning software. Mammalian bioassay and epidemiological studies that meet the PECO criteria after full-text review are briefly summarized using structured web-based extraction forms with respect to study design and health system(s) assessed. Extracted data is available in interactive visual formats and can be downloaded in open access formats. Methods for conducting study evaluation are also presented which is conducted on a case-by-case basis, depending on the usage of the SEM.

Use of systematic evidence maps within the US environmental protection agency (EPA) integrated risk information system (IRIS) program: Advancements to date and looking ahead.

Systematic evidence maps (SEMs) are increasingly used to inform decision-making and risk management priority-setting and to serve as problem formulation tools to refine the focus of questions that get addressed in full systematic reviews. Within the U.S. Environmental Protection Agency (EPA) Office of Research and Development (ORD) Integrated Risk Information System (IRIS), SEMs have been used to inform data gaps, determine the need for updated assessments, inform assessment priorities, and inform development of study evaluation considerations, among other uses. Increased utilization of SEMs across the environmental health field has the potential to increase transparency and efficiency for data gathering, problem formulation, read-across, and evidence surveillance. Use of the SEM templates published in the companion text (Thayer et al.) can promote harmonization in the environmental health community and create more opportunities for sharing extracted content.

'Omics in environmental epidemiological studies of chemical exposures: A systematic evidence map.

BACKGROUND: Systematic evidence maps are increasingly used to develop chemical risk assessments. These maps can provide an overview of available studies and relevant study information to be used for various research objectives and applications. Environmental epidemiological studies that examine the impact of chemical exposures on various 'omic profiles in human populations provide relevant mechanistic information and can be used for benchmark dose modeling to derive potential human health reference values. OBJECTIVES: To create a systematic evidence map of environmental epidemiological studies examining environmental contaminant exposures with 'omics in order to characterize the extent of available studies for future research needs. METHODS: Systematic review methods were used to search and screen the literature and included the use of machine learning methods to facilitate screening studies. The Populations, Exposures, Comparators and Outcomes (PECO) criteria were developed to identify and screen relevant studies. Studies that met the PECO criteria after full-text review were summarized with information such as study population, study design, sample size, exposure measurement, and 'omics analysis. RESULTS: Over 10,000 studies were identified from scientific databases. Screening processes were used to identify 84 studies considered PECO-relevant after full-text review. Various contaminants (e.g. phthalate, benzene, arsenic, etc.) were investigated in epidemiological studies that used one or more of the four 'omics of interest: epigenomics, transcriptomics, proteomics, and metabolomics . The epidemiological study designs that were used to explore single or integrated 'omic research questions with contaminant exposures were cohort studies, controlled trials, cross-sectional, and case-control studies. An interactive web-based systematic evidence map was created to display more study-related information. CONCLUSIONS: This systematic evidence map is a novel tool to visually characterize the available environmental epidemiological studies investigating contaminants and biological effects using 'omics technology and serves as a resource for investigators and allows for a range of applications in chemical research and risk assessment needs.

Harmonization of transcriptomic and methylomic analysis in environmental epidemiology studies for potential application in chemical risk assessment.

Recent efforts have posited the utility of transcriptomic-based approaches to understand chemical-related perturbations in the context of human health risk assessment. Epigenetic modification (e.g., DNA methylation) can influence gene expression changes and is known to occur as a molecular response to some chemical exposures. Characterization of these methylation events is critical to understand the molecular consequences of chemical exposures. In this context, a novel workflow was developed to interrogate publicly available epidemiological transcriptomic and methylomic data to identify relevant pathway level changes in response to chemical exposure, using inorganic arsenic as a case study. Gene Set Enrichment Analysis (GSEA) was used to identify causal methylation events that result in concomitant downstream transcriptional deregulation. This analysis demonstrated an unequal distribution of differentially methylated regions across the human genome. After mapping these events to known genes, significant enrichment of a subset of these pathways suggested that arsenic-mediated methylation may be both specific and non-specific. Parallel GSEA performed on matched transcriptomic samples determined that a substantially reduced subset of these pathways are enriched and that not all chemically-induced methylation results in a downstream alteration in gene expression. The resulting pathways were found to be representative of well-established molecular events known to occur in response to arsenic exposure. The harmonization of enriched transcriptional patterns with those identified from the methylomic platform promoted the characterization of plausibly causal molecular signaling events. The workflow described here enables significant gene and methylation-specific pathways to be identified from whole blood samples of individuals exposed to environmentally relevant chemical levels. As future efforts solidify specific causal relationships between these molecular events and relevant apical endpoints, this novel workflow could aid risk assessments by identifying molecular targets serving as biomarkers of hazard, informing mechanistic understanding, and characterizing dose ranges that promote relevant molecular/epigenetic signaling events occuring in response to chemical exposures.

Systematic Evidence Map for Over One Hundred and Fifty Per- and Polyfluoroalkyl Substances (PFAS).

BACKGROUND: Per- and polyfluoroalkyl substances (PFAS) are a large class of synthetic (man-made) chemicals widely used in consumer products and industrial processes. Thousands of distinct PFAS exist in commerce. The 2019 U.S. Environmental Protection Agency (U.S. EPA) Per- and Polyfluoroalkyl Substances (PFAS) Action Plan outlines a multiprogram national research plan to address the challenge of PFAS. One component of this strategy involves the use of systematic evidence map (SEM) approaches to characterize the evidence base for hundreds of PFAS. OBJECTIVE: SEM methods were used to summarize available epidemiological and animal bioassay evidence for a set of ∼150 PFAS that were prioritized in 2019 by the U.S. EPA's Center for Computational Toxicology and Exposure (CCTE) for in vitro toxicity and toxicokinetic assay testing. METHODS: Systematic review methods were used to identify and screen literature using manual review and machine-learning software. The Populations, Exposures, Comparators, and Outcomes (PECO) criteria were kept broad to identify mammalian animal bioassay and epidemiological studies that could inform human hazard identification. A variety of supplemental content was also tracked, including information on in vitro model systems; exposure measurement-only studies in humans; and absorption, distribution, metabolism, and excretion (ADME). Animal bioassay and epidemiology studies meeting PECO criteria were summarized with respect to study design, and health system(s) were assessed. Because animal bioassay studies with ≥21-d exposure duration (or reproductive/developmental study design) were most useful to CCTE analyses, these studies underwent study evaluation and detailed data extraction. All data extraction is publicly available online as interactive visuals with downloadable metadata. RESULTS: More than 40,000 studies were identified from scientific databases. Screening processes identified 44 animal and 148 epidemiology studies from the peer-reviewed literature and 95 animal and 50 epidemiology studies from gray literature that met PECO criteria. Epidemiological evidence (available for 15 PFAS) mostly assessed the reproductive, endocrine, developmental, metabolic, cardiovascular, and immune systems. Animal evidence (available for 40 PFAS) commonly assessed effects in the reproductive, developmental, urinary, immunological, and hepatic systems. Overall, 45 PFAS had evidence across animal and epidemiology data streams. DISCUSSION: Many of the ∼150 PFAS were data poor. Epidemiological and animal evidence were lacking for most of the PFAS included in our search. By disseminating this information, we hope to facilitate additional assessment work by providing the initial scoping literature survey and identifying key research needs. Future research on data-poor PFAS will help support a more complete understanding of the potential health effects from PFAS exposures. https://doi.org/10.1289/EHP10343.

Total Blood Mercury Predicts Methylmercury Exposure in Fish and Shellfish Consumers.

Many studies evaluating methylmercury (MeHg) toxicity rely on whole blood total mercury (THg) measurements to estimate MeHg exposure. However, whole blood THg includes other forms of mercury (Hg), such as inorganic Hg, which have different exposure sources and toxicological effects than MeHg. Therefore, estimating the whole blood MeHg/THg ratio is critical to predicting MeHg exposure and, subsequently, efforts to establish an exposure-response relationship for use in risk assessment. A large, representative dataset (National Health and Nutrition Examination Survey (NHANES) 2011-2016) was used to determine the whole blood MeHg/THg ratio among (a) self-reported fish and shellfish consumers, ≥ 15 years of age (the "full adult" population (N = 5268 training dataset; N = 2336 test dataset)) and (b) female fish and shellfish consumers, 15-44 years of age (the "women of reproductive age" population (N = 1285 training dataset; N = 560 test dataset)). Unadjusted and adjusted linear and spline models with direct measurements for both THg and MeHg were evaluated. The mean whole blood MeHg/THg ratio was 0.75 (95% confidence interval (CI): 0.74, 0.75). This ratio was significantly higher among those with higher THg concentrations. All models exhibited excellent fit (adjusted R2 from 0.957 to 0.982). Performance was slightly improved in spline versus linear models. For the full adult population and women of reproductive age, the unadjusted spline model predicted whole blood MeHg concentrations of 5.65 µg/L and 5.55 µg/L, respectively, when the THg concentration was 5.80 µg/L. These results suggest that whole blood THg is a good predictor of whole blood MeHg among fish and shellfish consumers.

Integrated Risk Information System (IRIS) response to "Assessing risk of bias in human environmental epidemiology studies using three tools: different conclusions from different tools".

"Assessing risk of bias in human environmental epidemiology studies using three tools: different conclusions from different tools," a recent publication in this journal, applied the study evaluation approach developed by the U.S. Environmental Protection Agency's Integrated Risk Information System (IRIS), as well as other approaches, to a set of studies examining polybrominated diphenyl ethers (PBDEs) and neurodevelopment. They concluded that use of the IRIS approach resulted in exclusion of studies, which would lead to hazard conclusions based on an incomplete body of evidence. As scientists in the IRIS program, we support the comparison of approaches to improve systematic review methods for environmental exposures; however, we believe the IRIS approach was misrepresented. In this letter, we demonstrate that the ratings attributed to the IRIS approach were not consistent with our own application of the tool. We also clarify the use of studies rated as "low confidence" and the use of an overall study confidence rating in our systematic reviews. In conclusion, the IRIS study evaluation approach is a transparent method to inform certainty in our evidence synthesis decisions and ensures consistency in the development of IRIS health assessments.

Improving the quality of toxicology and environmental health systematic reviews: What journal editors can do.

Systematic reviews are fast increasing in prevalence in the toxicology and environmental health literature. However, how well these complex research projects are being conducted and reported is unclear. Since editors have an essential role in ensuring the scientific quality of manuscripts being published in their journals, a workshop was convened where editors, systematic review practitioners, and research quality control experts could discuss what editors can do to ensure the systematic reviews they publish are of sufficient scientific quality. Interventions were explored along four themes: setting standards; reviewing protocols; optimizing editorial workflows; and measuring the effectiveness of editorial interventions. In total, 58 editorial interventions were proposed. Of these, 26 were shortlisted for being potentially effective, and 5 were prioritized as short-term actions that editors could relatively easily take to improve the quality of published systematic reviews. Recent progress in improving systematic reviews is summarized, and outstanding challenges to further progress are highlighted.

Challenges and recommendations on the conduct of systematic reviews of observational epidemiologic studies in environmental and occupational health.

Systematic reviews are powerful tools for drawing causal inference for evidence-based decision-making. Published systematic reviews and meta-analyses of environmental and occupational epidemiology studies have increased dramatically in recent years; however, the quality and utility of published reviews are variable. Most methodologies were adapted from clinical epidemiology and have not been adequately modified to evaluate and integrate evidence from observational epidemiology studies assessing environmental and occupational hazards, especially in evaluating the quality of exposure assessments. Although many reviews conduct a systematic and transparent assessment for the potential for bias, they are often deficient in subsequently integrating across a body of evidence. A cohesive review considers the impact of the direction and magnitude of potential biases on the results, systematically evaluates important scientific issues such as study sensitivity and effect modifiers, identifies how different studies complement each other, and assesses other potential sources of heterogeneity. Given these challenges of conducting informative systematic reviews of observational studies, we provide a series of specific recommendations based on practical examples for cohesive evidence integration to reach an overall conclusion on a body of evidence to better support policy making in public health.

Phthalate exposure and neurodevelopment: A systematic review and meta-analysis of human epidemiological evidence.

OBJECTIVE: We performed a systematic review of the epidemiology literature to identify the neurodevelopmental effects associated with phthalate exposure. DATA SOURCES AND STUDY ELIGIBILITY CRITERIA: Six phthalates were included in the review: di(2-ethylhexyl) phthalate (DEHP), diisononyl phthalate (DINP), dibutyl phthalate (DBP), diisobutyl phthalate (DIBP), butyl benzyl phthalate (BBP), and diethyl phthalate (DEP). The initial literature search (of PubMed, Web of Science, and Toxline) included all studies of neurodevelopmental effects in humans, and outcomes were selected for full systematic review based on data availability. STUDY EVALUATION AND SYNTHESIS METHODS: Studies of neurodevelopmental effects were evaluated using criteria defined a priori for risk of bias and sensitivity by two reviewers using a domain-based approach. Evidence was synthesized by outcome and phthalate and strength of evidence was summarized using a structured framework. For studies of cognition and motor effects in children ≤4 years old, a random effects meta-analysis was performed. RESULTS: The primary outcomes reviewed here are (number of studies in parentheses): cognition (14), motor effects (9), behavior, including attention deficit hyperactivity disorder (20), infant behavior (3), and social behavior, including autism spectrum disorder (7). For each phthalate/outcome combination, there was slight or indeterminate evidence of an association, with the exception of motor effects for BBP, which had moderate evidence. CONCLUSIONS AND IMPLICATIONS OF KEY FINDINGS: Overall, there is not a clear pattern of association between prenatal phthalate exposures and neurodevelopment. There are several possible reasons for the observed null associations related to exposure misclassification, periods of heightened susceptibility, sex-specific effects, and the effects of phthalate mixtures. Until these limitations are adequately addressed in the epidemiology literature, these findings should not be interpreted as evidence that there are no neurodevelopmental effects of phthalate exposure. The views expressed are those of the authors and do not necessarily represent the views or policies of the U.S. EPA.

Seafood, wine, rice, vegetables, and other food items associated with mercury biomarkers among seafood and non-seafood consumers: NHANES 2011-2012.

Fish/seafood consumption is a source of mercury; other dietary sources are not well described. This cross-sectional study used National Health and Nutrition Examination Survey (NHANES) 2011-2012 data. Participants self-reported consuming fish/seafood (N = 5427) or not (N = 1770) within the past 30 days. Whole blood total mercury (THg), methylmercury (MeHg), and urinary mercury (UHg) were determined. Diet was assessed using 24 h recall. Adjusted regression models predicted mercury biomarker concentrations with recent food consumption, while controlling for age, sex, education, and race/ethnicity. Geometric mean THg was 0.89 µg/L (95% confidence interval (CI): 0.78, 1.02) (seafood consumers) and 0.31 µg/L (95% CI: 0.28, 0.34) (non-seafood consumers); MeHg and UHg concentrations follow similar patterns. In adjusted regressions among seafood consumers, significant associations were observed between mercury biomarkers with multiple foods, including fish/seafood, wine, rice, vegetables/vegetable oil, liquor, and beans/nuts/soy. Among non-seafood consumers, higher THg was significantly associated with mixed rice dishes, vegetables/vegetable oil, liquor, and approached statistical significance with wine (p < 0.10); higher MeHg was significantly associated with wine and higher UHg was significantly associated with mixed rice dishes. Fish/seafood consumption is the strongest dietary predictor of mercury biomarker concentrations; however, consumption of wine, rice, vegetables/vegetable oil, or liquor may also contribute, especially among non-seafood consumers.

Use of the Adverse Outcome Pathway (AOP) framework to evaluate species concordance and human relevance of Dibutyl phthalate (DBP)-induced male reproductive toxicity.

Dibutyl phthalate (DBP) is a phthalate ester used as a plasticizer, and solvent. Studies using rats consistently report that DBP exposure disrupts normal development of the male reproductive system in part via inhibition of androgen synthesis. However, studies using xenograft models report that in human fetal testis DBP exposure is unlikely to impair testosterone synthesis. These results question the validity of the rat model for assessment of male reproductive effects caused by DBP. The Adverse Outcome Pathway (AOP) framework was used to evaluate the available evidence for DBP-induced toxicity to the male reproductive system. Three relevant biological elements were identified: 1) fetal rats are more sensitive than other rodents and human fetal xenografts to DBP-induced anti-androgenic effects, 2) DBP-induced androgen-independent adverse outcomes are conserved amongst different mammalian models and human fetal testis xenografts, and 3) DBP-induced anti-androgenic effects are conserved in different mammalian species when exposure occurs during postnatal life stages.

Development of outcome-specific criteria for study evaluation in systematic reviews of epidemiology studies.

INTRODUCTION AND OBJECTIVE: Systematic review tools that provide guidance on evaluating epidemiology studies are receiving increasing attention and support because their application facilitates improved quality of the review, consistency across reviewers, and transparency for readers. The U.S. Environmental Protection Agency's Integrated Risk Information System (IRIS) Program has developed an approach for systematic review of evidence of health effects from chemical exposures that includes structured approaches for literature search and screening, study evaluation, data extraction, and evidence synthesis and integration. This approach recognizes the need for developing outcome-specific criteria for study evaluation. Because studies are assessed at the outcome level, a study could be considered high quality for one investigated outcome, and low quality for another, due to differences in the outcome measures, analytic strategies, how relevant a certain bias is to the outcome, and how the exposure measure relates to the outcome. The objective of this paper is to illustrate the need for outcome-specific criteria in study evaluation or risk of bias evaluation, describe the process we used to develop the criteria, and summarize the resulting criteria. METHODS: We used a process of expert consultation to develop several sets of outcome-specific criteria to guide study reviewers, improve consistency, and ensure consideration of critical issues specific to the outcomes. The criteria were developed using the following domains: outcome assessment, exposure measurement (specifically timing of exposure in relation to outcome; other exposure measurement issues would be addressed in exposure-specific criteria), participant selection, confounding, analysis, and sensitivity (the study's ability to detect a true effect or hazard). RESULTS: We discuss the application of this process to pregnancy-related outcomes (preterm birth, spontaneous abortion), other reproductive-related outcomes (male reproductive hormones, sperm parameters, time to pregnancy, pubertal development), chronic disease (diabetes, insulin resistance), and acute or episodic conditions (asthma, allergies), and provide examples of the criteria developed. For each outcome the most influential methodological considerations are highlighted including biological sample collection and quality control, sensitivity and specificity of ascertainment tools, optimal timing for recruitment into the study (e.g., preconception, specific trimesters), the etiologically relevant window for exposure assessments, and important potential confounders. CONCLUSIONS: Outcome-specific criteria are an important part of a systematic review and will facilitate study evaluations by epidemiologists with experience in evaluating studies using systematic review methods who may not have extensive discipline-specific experience in the outcomes being reviewed.